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March  7, 200$ 

MEMORANDUM  FOR  UNDER  SECRETARY  OF  DEFENSE  FOR  ACQUISTION, 

TECHNOLOGY,  AND  LOGISTICS 
AUDITOR  GENERAL,  DEPARTMENT  OF  THE  ARMY 


SUBJECT:  Report  on  Implementation  of  Performance-Based  Logistics  for  the  Javelin 
Weapon  System  (Report  No.  D-2005-037) 


We  are  providing  this  report  for  information  and  use.  We  considered 
management  comments  on  a  draft  version  of  this  report  when  preparing  the  final  report. 
As  a  result  of  management  comments  and  further  discussions,  we  revised  finding  A  to 
clarify  our  intention  and  deleted  draft  Recommendation  A.  1 .  We  also  renumbered  draft 
Recommendations  A.2.a.  and  A.2.b.  to  A.l.  and  A.2. 

We  considered  management  comments  to  be  responsive  and  in  conformance  with 
the  requirements  of  DoD  Directive  7650.3.  Therefore,  additional  comments  are  not 
required. 

We  appreciate  the  courtesies  extended  to  the  staff.  Questions  should  be  directed 
to  Mr.  Robert  F.  Prinzbach  at  (703)  604-8907  (DSN  664-8907)  or  Mr.  Keith  A.  Yancey  at 
(703)  604-8774  (DSN  664-8774).  See  Appendix  C  for  the  report  distribution.  The  team 
members  are  listed  inside  the  back  cover. 

By  direction  of  the  Deputy  Inspector  General  for  Auditing: 


Acting  Assistant  Inspector  General 
for  Readiness  and  Logistics  Support 


Department  of  Defense  Office  of  Inspector  General 


Report  No.  D-2005-037  March  7,  2005 

(Project  No.  D2004LH-0047) 

Implementation  of  Performance-Based 
Logistics  for  the  Javelin  Weapon  System 


Executive  Summary 


Who  Should  Read  This  Report  and  Why?  DoD  personnel  and  Government 
contractors  who  are  responsible  for  implementing  performance-based  logistics  (PBL) 
should  read  this  report.  This  report  discusses  the  status  of  PBL  implementation  for  the 
Javelin  weapon  system. 

Background.  PBL  is  a  strategy  for  weapon  system  product  support  that  employs  the 
purchase  of  support  as  an  integrated  performance  package  designed  to  optimize  system 
readiness.  PBL  delineates  outcome  performance  goals  of  weapon  systems,  ensures  that 
responsibilities  are  assigned,  and  provides  incentives  for  attaining  those  goals  for  the  life 
of  the  weapon  system.  The  life-cycle  management  of  a  weapon  system  ensures  its 
reliability,  supportability,  and  total  ownership  cost.  PBL  is  the  DoD-preferred  approach 
for  providing  logistics  support  to  weapon  systems. 

The  Defense  Planning  Guidance  for  FYs  2003  through  2007  requires  that  each  Military 
Department  submit  a  plan  that  identifies  its  implementation  schedule  for  applying  PBL  to 
all  new  weapon  systems  and  all  Acquisition  Categoiy  I  and  II  fielded  systems.  As  of 
August  23,  2004,  the  Military  Departments  reported  257  systems,  sub-systems,  or 
components  as  having  implemented  PBL. 

The  Javelin  weapon  system,  an  Acquisition  Category  I  program,  is  a  medium-range, 
infrared-imaging,  fire  and  forget,  man-portable,  antitank  weapon  system  developed  for 
the  Aimy  and  the  Marine  Corps.  It  is  composed  of  a  tactical  round  and  a  command 
launch  unit.  The  Javelin  weapon  system  was  designed,  developed,  and  fabricated  by  the 
Raytheon  and  Lockheed  Martin  Javelin  Joint  Venture.  In  July  2002,  the  Army  reported 
that  PBL  had  been  implemented  for  the  Javelin  weapon  system. 

Results.  The  Aimy  reported  a  PBL  strategy  for  the  Javelin  weapon  system  to  the  Under 
Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics.  However,  the  strategy 
was  not  fully  implemented  as  described  in  the  Army’s  July  8,  2002,  PBL  implementation 
schedule.  As  a  result,  the  Aimy  might  not  be  realizing  the  benefits  of  PBL  for  the  Javelin 
weapon  system,  such  as  improved  readiness  and  decreased  maintenance  cost,  and  might 
be  overstating  its  PBL  progress.  The  Project  Manager,  Close  Combat  Weapon  Systems 
Project  Office,  as  the  project  management  office,  should  update  performance-based 
agreements  with  warfighters  and  modify  the  Javelin  weapon  system  life-cycle  contract  to 
incorporate  provisions  for  incentives  and  penalties  that  would  support  PBL  (finding  A). 

The  project  management  office’s  decision  to  award  the  life-cycle  contractor  support 
contract  to  the  Raytheon  and  Lockheed  Martin  Javelin  Joint  Venture  was  based  on  an 
economic  analysis  that  was  unsupported  and  incomplete.  As  a  result,  the  best  alternative 


for  life -cycle  support  of  the  Javelin  weapon  system  might  not  have  been  selected.  The 
Project  Manager,  Close  Combat  Weapon  Systems  Project  Office  should  update  the 
economic  analysis  and  incorporate  it  into  a  business  case  analysis  for  the  Javelin  weapon 
system.  Further,  based  on  the  revised  baseline  cost  estimate,  the  project  management 
office  should  reassess  logistics  support  strategies  for  the  Javelin  weapon  system  after  the 
Deputy  Assistant  Secretary  of  the  Army  (Cost  and  Economics)  validates  the  business 
case  analysis  (finding  B). 

The  recommendations  in  this  report,  if  implemented,  will  correct  the  material 
management  control  weaknesses  we  identified.  See  the  Findings  section  of  the  report  for 
the  detailed  recommendations. 

Management  Comments  and  Audit  Response.  The  Acting  Deputy  Under  Secretary  of 
Defense  (Logistics  and  Materiel  Readiness)  and  the  Army  disagreed  with  our  draft 
finding  that  the  Javelin  weapon  system  was  incorrectly  reported  as  PBL.  After  reviewing 
management  comments  and  further  discussion  with  personnel  from  the  Office  of  the 
Deputy  Under  Secretary  of  Defense  (Logistics  and  Materiel  Readiness),  we  revised  the 
finding  to  clarify  our  intention  and  deleted  the  draft  recommendation  that  the  Army  cease 
reporting  that  it  had  implemented  performance-based  logistics  for  the  Javelin  weapon 
system. 

The  Army  partially  concurred  with  our  recommendation  to  update  performance-based 
agreements  with  warfighters  and  stated  that  the  Army  is  developing  a  policy  on 
performance -based  agreements  with  the  intent  being  to  allow  flexibility  in  using  several 
existing  documents  to  satisfy  the  performance- based  agreement  requirement.  Once 
approved,  the  Javelin  project  manager  will  update  materiel  fielding  agreements  as 
necessary.  In  addition,  the  Army  partially  concurred  with  our  recommendation  to  modify 
the  life-cycle  contract  for  the  Javelin  weapon  system  and  stated  that  when  Javelin 
program  requirements  require  a  change  in  the  life-cycle  contractor  support  contract,  the 
project  management  office  will  reevaluate  incentives  and  penalties  in  the  contract. 

The  Amy  partially  concurred  with  our  recommendation  to  update  the  economic  analysis 
and  stated  that  the  economic  analysis  would  be  updated  to  a  business  case  analysis  when 
contract  situations  require  it.  The  Amy  concurred  with  our  recommendation  to  fully 
validate  the  business  case  analysis  for  the  Javelin  weapon  system  and  to  formally 
document  the  results  and  conclusions  of  that  validation. 

We  considered  management  comments  to  be  responsive.  See  the  Findings  section  of  the 
report  for  a  discussion  of  management  comments  and  the  Management  Comments 
section  for  the  complete  text  of  comments. 
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Background 


Performance-Based  Logistics.  Performance-based  logistics  (PBL)  is  the  DoD- 
preferred  approach  for  implementing  product  support.  PBL  is  a  strategy  for 
weapon  system  product  support  that  employs  the  purchase  of  support  as  an 
integrated  performance  package  designed  to  bring  increased  levels  of  system 
readiness.  PBL  describes  performance  goals  for  weapon  system  readiness  and 
encourages  the  creation  of  incentives  for  attaining  those  goals  through  clear  lines 
of  authority  and  responsibility.  PBL  delineates  outcome  performance  goals  of 
weapon  systems,  ensures  that  responsibilities  are  assigned,  and  provides 
incentives  for  attaining  those  goals  for  the  overall  life-cycle  management  of 
system  reliability,  supportability,  and  total  ownership  cost.  The  Under  Secretary 
of  Defense  for  Acquisition,  Technology,  and  Logistics  (USD[AT&L])  assigned 
the  Assistant  Deputy  Under  Secretary  of  Defense  (Logistics  Plans  and  Programs) 
the  responsibility  to  provide  oversight  for  the  implementation  of  PBL  within 
DoD. 

In  September  2001,  the  “Quadrennial  Defense  Review” 'mandated  implementation 
of  PBL  and  modern  business  systems  with  appropriate  metrics  to  compress  the 
supply  chain,  eliminate  steps  that  were  “non-value-added”  steps,  and  improve 
readiness  for  major  weapon  systems.  In  Department  of  Defense  Inspector 
General  Report  No.  D-2004-110,  “The  Military  Departments’  Implementation  of 
Performance-Based  Logistics  in  Support  of  Weapon  Systems,”  August  23,  2004, 
we  reported  that  the  Military  Departments  had  257  systems,  sub-systems,  or 
components  reported  as  having  implemented  PBL. 

Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
Memorandum.  USD(AT&L)  issued  a  memorandum,  “Performance-Based 
Logistics,”  February  13,  2002,  which  states  that  the  Defense  Planning  Guidance 
for  FY s  2003  through  2007  requires  that  each  Military  Department  submit  a  plan 
that  identifies  its  implementation  schedule  for  applying  PBL  to  all  new  weapon 
systems  and  all  Acquisition  Category2 1  and  II  fielded  systems.  The  Military 
Departments  were  to  prepare  and  submit  their  PBL  plans  to  USD(AT&L)  by 
May  1,  2002,  for  review  and  monitoring.  The  Assistant  Secretary  of  the  Army 
(Acquisition,  Logistics,  and  Technology)  submitted  the  Javelin  PBL  strategy  on 
July  8,  2002. 

DoD  Directive  5000.1.  DoD  Directive  5000.1,  “The  Defense  Acquisition 
System,”  May  12,  2003,  states  that  “PMs  [program  managers]  shall  develop  and 
implement  performance-based  logistics  strategies  that  optimize  total  system 
availability  while  minimizing  cost  and  logistics  footprint.”  The  Directive  also 


1  The  Quadrennial  Defense  Review  serves  as  the  overall  strategic  planning  document  for  DoD,  as  required 
by  Public  Law  103-62,  “Government  Performance  and  Results  Act  of  1993.” 

2  The  acquisition  category  determines  an  acquisition  program’s  level  of  review,  decision  authority,  and 
applicable  procedures.  Acquisition  Category  I  programs  are  acquisition  programs  with  an  estimated  total 
expenditure  for  research,  development,  test,  and  evaluation  of  more  than  $365  million  or  procurement  of 
more  than  $2.19  billion.  Acquisition  Category  II  programs  have  an  estimated  total  expenditure  for 
research,  development,  test,  and  evaluation  of  $365  million  or  less  but  more  than  $140  million  or 
procurement  of  $2.19  billion  or  less  but  more  than  $660  million. 
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requires  that  the  program  manager  for  a  system  be  the  single  point  of 
accountability  for  accomplishing  program  objectives  for  the  system’s  life-cycle 
management,  including  sustainment. 

Spectrum  of  PBL  Strategies.  According  to  the  Office  of  the  Deputy  Under 
Secretary  of  Defense  (Logistics  and  Materiel  Readiness)  (DUSD[L&MR])  guide, 
“Product  Support  for  the  21st  Century:  A  Program  Manager’s  Guide  to  Buying 
Performance”  (Product  Support  Guide),  November  6,  2001,  a  PBL  strategy  seeks 
to  maintain  the  appropriate  level  of  flexibility  and  agility  to  evolve  with 
technological  advances  and  warfighters’  requirements.  The  strategy  should  be 
designed  to  balance  two  major  objectives:  logistics  support  and  improvements  in 
cost-effectiveness  of  logistics  products  and  services.  PBL  strategies  vaiy 
depending  on  the  age  of  the  system,  existing  support  infrastructure,  organic 
(DoD)  and  commercial  capabilities,  and  legislative  and  regulatory  constraints. 
PBL  strategies  include  assigning  total  system  support  responsibility,  partnering 
contractors  with  DoD  depots,  and  establishing  performance-based  agreements 
(PBAs)  with  operational  commands  and  organizations  (warfighters). 

J avelin  Weapon  System.  The  Javelin  weapon  system  is  a  medium- range, 
infrared-imaging,  fire  and  forget,  man-portable,  antitank  weapon  system 
developed  for  the  Army  and  the  Marine  Coips.  It  is  composed  of  a  tactical  round 
and  a  command  launch  unit.  The  command  launch  unit  is  used  for  battlefield 
surveillance,  target  acquisition,  missile  launch,  and  damage  assessment.  The 
Javelin  weapon  system  was  designed,  developed,  and  fabricated  by  Raytheon  and 
Lockheed  Martin  Javelin  Joint  Venture  (JV  Contractor).  Since  the  initial  fielding 
of  the  Javelin  weapon  system  in  1996,  the  JV  Contractor  has  been  responsible  for 
maintenance  of  the  Javelin  weapon  system  through  various  interim  contractor 
support  contracts.  The  Javelin  weapon  system  is  an  Acquisition  Category  I 
program.  The  project  management  office  (PMO)  for  the  Javelin  weapon  system 
is  the  Close  Combat  Weapon  Systems  Project  Office,  located  at  the  Aviation  and 
Missile  Command,  Redstone  Arsenal,  Huntsville,  Alabama.  The  Program 
Executive  Office,  Tactical  Missiles,  Aviation  and  Missile  Command,  provides 
oversight  of  the  PMO. 

Reporting  of  J  avelin  PBL.  The  Assistant  Secretary  of  the  Aimy  (Acquisition, 
Logistics,  and  Technology)  reported  to  USD(AT&L)  that  PBL  for  the  Javelin 
weapon  system  had  been  implemented.  In  a  memorandum  to  USD(AT&L), 
“Performance-Based  Logistics,”  July  8,  2002,  the  Assistant  Secretary  stated  that, 
in  accordance  with  the  USD(AT&L)  February  13,  2002,  memorandum, 
acquisition  managers  of  all  Acquisition  Category  I  and  II  programs  had  assessed 
their  programs  for  implementation  of  PBL.  Further,  the  Assistant  Secretary 
provided  the  Aimy’s  PBL  implementation  schedule  to  USD(AT&L),  which 
showed  that  PBL  for  the  Javelin  weapon  system  had  been  implemented.  The 
implementation  schedule  also  showed  the  PBL  strategy  for  the  Javelin  weapon 
system,  which  included  integrated  product  support  provider  (such  as  a  prime 
contractor  or  a  logistics  command),  PBAs  with  warfighters,  PBAs  with  organic 
(DoD  depots)  and  commercial  providers,  partnering,  and  contract  incentives  and 
penalties. 
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Objectives 


Our  overall  audit  objective  was  to  evaluate  the  implementation  of  PBL  for  the 
Javelin  weapon  system.  We  also  reviewed  the  management  control  program 
related  to  the  audit  objective.  See  Appendix  A  for  a  discussion  of  our  scope  and 
methodology  and  our  review  of  the  management  control  program.  See 
Appendix  B  for  prior  coverage  related  to  the  objectives. 
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A.  Javelin  Performance-Based  Logistics 

The  Army  reported  a  PBL  strategy  for  the  Javelin  weapon  system  to 
USD(AT&L).  However,  the  strategy  was  not  fully  implemented  because 
PBAs  were  not  updated  to  reflect  warfighter  requirements  and 
performance  incentives  and  penalties  were  not  incorporated  into  the  life- 
cycle  contractor  support  (LCCS) 3  contract.  As  a  result,  the  Army  might 
not  be  realizing  the  benefits  of  PBL  for  the  Javelin  weapon  system,  such 
as  improved  readiness  and  decreased  maintenance  cost,  and  might  be 
overstating  its  PBL  progress 


Federal  Acquisition  Regulation 


According  to  Federal  Acquisition  Regulation  Part  37.6,  “Performance  Based 
Contracting,”  August  25,  2003,  to  the  maximum  extent  possible,  performance 
incentives,  either  positive  or  negative  or  both,  are  to  be  incorporated  into  a 
contract  to  encourage  contractors  to  increase  efficiency  and  maximize 
performance.  Performance-based  contracts  describe  the  requirements  in  terms  of 
results  required  and  include  measurable  performance  standards.  Performance- 
based  contracts  also  include  incentives  for  superior  performance  and  specific 
procedures  for  reducing  payment  to  the  contractor  when  services  are  not 
performed  or  do  not  meet  contract  requirements. 


Reporting  Performance-Based  Logistics 


The  Army  reported  a  PBL  strategy  for  the  Javelin  weapon  system  to 
USD(AT&L).  The  PMO  first  reported  the  Javelin  weapon  system  as  having 
implemented  PBL  in  its  FY  2002  quarterly  report  to  the  Assistant  Secretary  of 
the  Army  (Acquisition,  Logistics,  and  Technology).  The  quarterly  report  showed 
Javelin  PBL  as  having  been  implemented  in  1996,  which  was  before  DoD 
established  a  PBL  requirement.  The  Assistant  Secretary  reported  in  his  July  8, 
2002,  implementation  schedule  to  USD(AT&L)  that  the  Javelin  weapon  system 
had  implemented  PBL.  In  this  schedule,  the  Assistant  Secretary  described  his 
strategy  for  implementing  PBL  for  the  Javelin  weapon  system,  which  called  for 
an  integrated  product  support  provider,  PBAs  with  warfighters,  PBAs  with  DoD 
depots  and  commercial  providers,  partnering  arrangements,  and  contract 
incentives  and  penalties. 


3  LCCS  is  a  method  of  providing  all  or  part  of  a  system’s  logistics  support  by  contract,  with  the  intention  of 
continuing  that  support  throughout  the  system’s  life  cycle. 
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Implementation  of  Performance-Based  Logistics 


The  Javelin  PBL  strategy  had  not  been  fully  implemented  because  PBAs  had  not 
been  updated  to  reflect  warfighter  requirements  and  performance  incentives  and 
penalties,  had  not  been  incorporated  into  the  LCCS  contract.  The  implementation 
schedule  that  the  Army  provided  to  USD(AT&L)  on  July  8,  2002,  reported  that 
the  PBL  strategy  for  the  Javelin  weapon  system,  including  PBAs  with  warfighters 
and  contract  incentives  and  penalties,  had  been  implemented.  As  of 
August  23,  2004,  the  Army  had  not  taken  action  to  incorporate  warfighters’ 
requirements  into  a  PBA  and  the  LCCS  contract  did  not  include  provisions  for 
performance  incentives  and  penalties.  Therefore,  the  Army  might  not  be  realizing 
the  benefits  of  PBL  for  the  Javelin  weapon  system,  such  as  improved  readiness 
and  decreased  maintenance  cost,  and  might  be  overstating  its  PBL  progress. 

Performance-Based  Agreements.  The  PMO  had  not  updated  PBAs  with 
warfighters  to  reflect  PBL  performance  goals  and  objectives  and  to  establish  a 
target  price  based  on  a  desired  level  of  performance.  According  to  the  Product 
Support  Guide,  a  written  performance  agreement  between  the  program  manager 
and  the  warfighter  is  the  centerpiece  of  the  program  manager’s  overall  PBL 
support  strategy.  Typically,  a  PBA  identifies  outcome  performance  goals  and 
objectives,  such  as  availability  and  cost,  and  establishes  a  target  price  based  on 
the  desired  level  of  performance.  Program  managers  and  warfighters  should  work 
together  to  determine  what  is  reasonable  and  attainable  given  the  state  of 
technology  and  resources.  The  Product  Support  Guide  further  states  that  reaching 
an  understanding  of  what  the  warfighter  wants  in  terms  of  performance  is 
essential  to  the  program  manager’s  ability  to  develop  a  meaningful  support 
strategy. 

The  PMO  had  established  materiel  fielding  agreements  for  the  Javelin  weapon 
system  with  several  warfighting  commands,  such  as  the  U.S.  Army  Europe,  the 
U.S.  Army  Forces  Command,  and  the  U.S.  Army  Special  Operations  Command. 
Those  fielding  agreements  were  established  as  far  back  as  1996,  which  predates 
PBL.  The  PMO  considered  those  fielding  agreements  as  the  warfighter  PBAs 
required  for  PBL  implementation.  However,  those  fielding  agreements  lacked 
PBL  requirements  such  as  PBL  performance  goals  and  objectives  and  a  target 
price  based  on  a  desired  level  of  performance.  The  purpose  of  a  materiel  fielding 
agreement  is  to  document  the  warfighters’  concurrence  of  the  administrative  and 
logistics  support  concept  for  the  fielding  of  a  weapon  system  and  not  to  set  a 
target  price  and  desired  level  of  performance.  For  example,  warfighter  and  PMO 
responsibilities  in  those  Javelin  fielding  agreements  included  staging  and  shipping 
of  fielding  packages,  conducting  joint  inventory  and  transfer  of  accountable 
property,  and  designating  a  point  of  contact.  As  of  August  23,  2004,  those 
agreements  had  not  been  updated  to  reflect  PBL  requirements,  which  is  almost 
3  years  after  PBL  requirements  were  issued  in  the  November  6,  2001,  Product 
Support  Guide. 

Performance  Incentives  and  Penalties.  The  LCCS  contract  did  not  contain 
provisions  for  performance  incentives  and  penalties  that  would  support  PBL 
implementation  and  motivate  the  JV  Contractor  to  attain  desired  levels  of  weapon 
system  performance.  Federal  Acquisition  Regulation  Part  37.6  requires 
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performance  incentives  to  be  incorporated  into  contracts  to  encourage  contractors 
to  increase  efficiency  and  maximize  performance.  The  Product  Support  Guide 
states  that  incentives  should  motivate  the  contractor  to  achieve  performance  levels 
of  the  highest  quality  consistent  with  economic  efficiency.  The  Product  Support 
Guide  also  references  the  USD(AT&L)  “Guide  to  Incentive  Strategies  for 
Defense  Acquisitions,”  January  2001,  which  states  that  establishing  incentives 
provides  “the  necessary  framework  and  tools  with  which  to  effectively  structure 
contractual  incentives  to  achieve  overall  best  value  as  part  of  a  successful 
business  relationship.” 

On  January  25,  2004,  the  Aviation  and  Missile  Command  awarded  a  1-year, 
firm-fixed-price  LCCS  contract  (W31P4Q-04-C-0046)  to  the  JV  Contractor  for 
maintenance  and  repair  of  the  Javelin  weapon  system.  The  contract  had  9  option 
years.  The  contract  included  fielding,  maintenance  support,  and  training  for 
Army,  Army  National  Guard,  Marine  Coips,  and  foreign  military  sales  customers. 

The  LCCS  contract  requires  the  contractor  to  maintain  a  90-percent  operational 
readiness  rate  and  a  10-day  repair  turnaround  time.  However,  the  contract  does 
not  contain  either  positive  or  negative  performance  incentives  tied  to  those 
requirements.  The  preferred  PBL  contracting  approach  is  to  use  long-term 
contracts  with  incentives  tied  to  performance.  Incorporating  incentives  and 
penalties  into  the  LCCS  contract  would  support  PBL  implementation  and  would 
encourage  the  JV  Contractor  to  optimize  performance  levels  for  the  Javelin 
weapon  system.  Because  the  PMO  had  not  fully  implemented  the  PBL  strategy 
for  the  Javelin  weapon  system,  the  Army  might  not  be  realizing  the  benefits  of 
PBL  for  the  Javelin  weapon  system,  such  as  improved  readiness  and  decreased 
maintenance  cost,  and  may  be  overstating  its  PBL  progress. 


Management  Comments  on  the  Finding  and  Audit  Response 


Management  Comments.  The  Acting  DUSD(L&MR)  and  the  Aimy  disagreed 
with  the  draft  report  finding  that  the  Javelin  weapon  system  was  incorrectly 
reported  as  PBL.  The  Acting  DUSD(L&MR)  and  the  Aimy  stated  that  the 
Javelin  LCCS  program  is  considered  to  be  a  PBL  program  because  the  support 
arrangement  is  based  on  performance  outcomes.  See  the  Management  Comments 
section  of  the  report  for  the  complete  text  of  the  comments. 

Audit  Response.  After  reviewing  management  comments  and  further  discussion 
with  personnel  from  the  Office  of  the  DUSD(L&MR),  we  agreed  that  the  Aimy 
was  not  incorrect  in  reporting  the  Javelin  as  having  implemented  PBL  and  revised 
finding  A  accordingly.  We  also  revised  finding  A  to  reflect  that  the  Army’s 
reported  strategy  for  implementing  PBL  for  the  Javelin  weapon  system  had  not 
been  fully  implemented. 
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Recommendations,  Management  Comments,  and  Audit 
Response 


Deleted  and  Renumbered  Recommendations.  As  a  result  of  management 
comments  and  further  discussion  with  personnel  from  the  Office  of  the 
DUSD(L&MR)  we  deleted  draft  Recommendation  A.l.  and  renumbered  draft 
Recommendations  A.2.a.  and  A.2.b.  as  A.l.  and  A.2.,  respectively.  We  also 
revised  the  finding  discussion  accordingly. 

A.  We  recommend  that  the  Project  Manager,  Close  Combat  Weapon 
Systems  Project  Office: 

1.  Update  performance-based  agreements  with  warfighters  using  the 
Javelin  weapon  system. 

Management  Comments.  The  Deputy  Assistant  Secretary  of  the  Army 
(Integrated  Logistics  Support)  (DASA[ILS])  provided  the  Army’s  comments  on 
finding  A.  He  stated  in  further  discussion  that  his  comments  also  incorporated 
comments  from  the  Project  Manager,  Close  Combat  Weapon  Systems  Project 
Office.  The  DASA(ILS)  partially  concurred,  stating  that  the  Army  considers 
materiel  fielding  agreements  to  be  an  acceptable  form  for  a  PBA  and  that  the 
Army  is  developing  a  policy  on  PBAs.  The  intent  of  the  policy  is  to  allow 
flexibility  in  using  several  existing  documents  to  satisfy  the  PBA  requirement. 
Once  the  policy  is  approved  and  disseminated,  the  Javelin  project  manager  will 
update  materiel  fielding  agreements  as  necessary. 

Audit  Response.  We  consider  the  Aimy  comments  to  be  responsive.  We  agree 
that  materiel  fielding  agreements  can  be  an  acceptable  form  of  a  PBA.  However, 
to  ensure  that  the  Javelin  weapon  system  obtains  the  desired  level  of  performance 
required  by  the  warfighter,  current  materiel  fielding  agreements  need  to  be 
updated  or  PBAs  need  to  be  developed  to  ensure  that  the  requirements  in  the 
current  LCCS  contract  of  a  90-percent  operational  readiness  rate  meets  the 
performance  requirements  of  warfighters  using  the  Javelin  weapon  system. 

2.  Modify  the  life-cycle  support  contract  for  the  Javelin  weapon 
system  to  incorporate  provisions  for  performance  incentives  and  penalties 
that  would  support  performance-based  logistics  and  motivate  the  contractor 
to  attain  desired  levels  of  performance  for  the  Javelin  weapon  system. 

Management  Comments.  The  DASA(ILS)  partially  concurred,  stating  that 
contract  incentives  and  penalties  are  the  correct  mechanism  to  use  in  PBL 
contracts;  however,  he  also  stated  that  they  are  not  mandatoiy.  In  addition,  the 
DASA(ILS)  stated  that  when  Javelin  program  requirements  require  a  change  in 
the  LCCS  contract,  the  PMO  will  reevaluate  incentives  and  penalties  included  in 
the  contract.  The  Acting  DUSD(L&MR)  stated  that  the  Army  should  incorporate 
incentives  and  penalties  into  the  PBL  arrangement,  but  only  when  it  is  prudent  to 
do  so. 

Audit  Response.  Although  the  DASA(ILS)  partially  concurred,  we  consider  the 
comments  to  be  responsive. 
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B.  Economic  Analysis 

The  PMO  decision  to  award  the  LCCS  contract  to  the  JV  Contractor  was 
based  on  an  economic  analysis  (EA)  that  was  unsupported  and  incomplete. 
The  EA  was  unsupported  and  incomplete  because  the  PMO  and  the  Office 
of  the  Deputy  Assistant  Secretary  of  the  Army  (Cost  and  Economics) 
(ODASA-CE)4  did  not  adhere  to  requirements  for  preparing  and  validating 
the  EA.  Specifically,  the  PMO  did  not  ensure  that  the  EA  had  a 
development  plan,  had  a  clear  audit  trail,  and  had  been  updated  with  key 
cost  elements.  Further,  ODASA-CE  had  not  fully  validated  the  EA.  As  a 
result,  the  best  alternative  for  the  life-cycle  support  of  the  Javelin  weapon 
system  might  not  have  been  selected. 


Army  Economic  Analysis  Manual 


The  Army  Economic  Analysis  Manual  (EA  Manual),  February  2001,  states  that 
the  EA  process  is  a  systematic  approach  to  identify,  analyze,  and  compare  costs 
and  benefits  of  alternative  courses  of  action  to  achieve  a  given  set  of  objectives. 
The  process  is  used  to  determine  the  most  efficient  and  effective  use  of  resources. 
An  EA  is  required  for  all  ongoing  programs  and  must  be  forwarded  to  Army 
headquarters  for  approval  when  there  is  a  choice  between  two  or  more 
alternatives. 

The  EA  Manual  requires  that  ongoing  programs  be  periodically  assessed  for  their 
cost-effectiveness,  which  requires  that  the  EA  be  regularly  updated.  Before 
preparing  an  EA,  the  program  manager  should  prepare  a  detailed  EA  development 
plan  and  provide  it  to  decision  makers  and  other  participants  in  the  review  and 
validation  process  of  the  EA.  According  to  the  EA  Manual,  all  EAs  require 
proper  validation  for  currency,  reasonableness,  completeness,  and  compliance. 
EAs  requiring  Army  headquarters  approval  should  have  a  thorough  validation, 
consisting  of  a  comprehensive  review  of  all  costs  and  benefits,  with  a  formally 
documented  report.  ODASA-CE,  an  agency  under  the  Assistant  Secretary  of  the 
Army  (Financial  Management  and  Comptroller),  is  the  proponent  for  validating 
the  EA. 


Establishment  of  an  Economic  Analysis 


The  decision  to  award  the  LCCS  contract  to  the  JV  Contractor  was  based  on  an 
EA  that  was  unsupported  and  incomplete.  In  September  2001,  the  PMO  prepared 
the  “Javelin  Life  Cycle  Support  Concept  Economic  Analysis”  for  the  Program 
Executive  Office,  Tactical  Missiles,  Aviation  and  Missile  Command.  The  EA 
was  in  response  to  a  USD(AT&L)  request  to  provide  a  plan  to  implement  Javelin 
LCCS.  The  EA  compared  the  cost  of  an  organic  (DoD)  life-cycle  support  concept 


4  In  our  draft  report,  ODASA-CE  was  referred  to  as  the  Army  Cost  and  Economic  Analysis  Center 
(CEAC). 
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with  the  cost  of  a  contractor  life-cycle  support  concept  for  the  Javelin  weapon 
system.  The  EA  compared  those  costs  over  a  period  of  24  years.  The  PMO 
concluded  that  the  EA  showed  no  clear  winner  based  on  total  cost.  In  addition, 
the  PMO  concluded  that,  in  all  cases,  an  LCCS-based  alternative  was  the  lowest 
cost.  On  October  4,  2001,  ODASA-CE  signed  the  memo  stating  that  it  concurred 
with  the  EA.  On  March  13,  2002,  the  Assistant  Secretary  of  the  Army 
(Acquisition,  Logistics,  and  Technology)  approved  the  recommendation  to 
implement  LCCS.  On  January  25,  2004,  2  years  after  ODASA-CE  concurred 
with  the  EA,  an  LCCS  contract,  based  on  the  September  2001  EA,  was  awarded 
to  the  JV  Contractor. 


Economic  Analysis  Guidelines 


The  EA  was  unsupported  and  incomplete  because  the  PMO  and  ODASA-CE  did 
not  adhere  to  requirements  for  preparing  and  validating  the  EA.  Specifically,  the 
PMO  did  not  ensure  the  EA  had  a  development  plan,  had  a  clear  audit  trail,  and 
had  been  updated  with  key  cost  elements.  Further,  ODASA-CE  had  not 
performed  a  comprehensive  review  of  the  EA  supporting  documentation  to  fully 
validate  the  EA. 

Economic  Analysis  Development  Plan.  The  PMO  did  not  prepare  an  EA 
development  plan  before  developing  the  EA  for  the  Javelin  weapon  system.  An 
EA  development  plan  consists  of  the  weapon  system’s  mission,  purpose, 
constraints,  assumptions,  cost  elements,  system  description,  and  schedules.  For  a 
project  of  high  dollar  value,  the  EA  development  plan  should  be  relatively 
detailed  and  should  be  provided  to  the  decision  maker  and  other  participants  in 
the  review  and  validation  process  before  the  analysis  is  performed.  PMO  officials 
stated  that  an  EA  development  plan  was  not  needed  because  it  was  clear  what  the 
objective  for  the  Javelin  weapon  system  was  when  they  compared  other 
alternatives  for  life-cycle  support.  Also,  because  the  PMO  held  regular  briefings 
that  discussed  ongoing  Javelin  weapon  system  issues,  personnel  were  fully  aware 
of  what  the  plans  were.  As  a  result  of  not  having  an  EA  development  plan,  we 
were  not  able  to  fully  assess  whether  the  PMO  sufficiently  accomplished  the  EA 
objective  for  the  Javelin  weapon  system.  ODASA-CE  officials  also  stated  that  an 
EA  development  plan  may  have  helped  them  in  fully  validating  the  EA. 

Audit  Trail.  The  PMO  did  not  maintain  an  adequate  audit  trail  to  support  key 
cost  elements  in  the  EA.  The  EA  Manual  states  that  there  must  be  sufficient 
documentation  of  all  assumptions,  costs,  methodology,  results,  and  data  to  enable 
a  person  unfamiliar  with  the  project  to  arrive  at  the  same  conclusions  as  the 
person  who  prepared  the  EA.  The  acceptance  of  the  EA  depends  on  the 
credibility  of  the  cost  estimates;  therefore,  the  analyst  must  document  data 
sources,  provide  the  derivation  of  all  costs,  and  maintain  a  clear  audit  trail. 
According  to  the  EA  Manual,  it  is  of  paramount  importance  to  maintain  an 
adequate  audit  trail  to  support  the  EA.  The  Javelin  PMO  did  not  have  detailed 
documentation  to  support  cost  and  benefit  elements  of  the  EA.  Having  detailed 
documentation  would  have  allowed  us  to  properly  assess  the  EA.  For  example, 
PMO  officials  regularly  cited  the  “Automated  Cost  Estimating  Integrated  Tools” 
model  for  supporting  documentation  of  the  EA.  The  model  is  a  system  used  to 
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standardize  and  simplify  the  process  of  estimating  the  life-cycle  costs  to  be 
included  in  an  EA.  However,  the  PMO  did  not  have  the  source  documentation  to 
support  cost  estimates,  such  as  overhead,  used  in  the  model.  As  a  result,  we  were 
not  able  to  arrive  at  the  same  conclusion  as  the  PMO  that  an  LCCS-based 
alternative  was  the  lowest  cost  for  the  Javelin  weapon  system. 

Updating  the  EA.  The  PMO  did  not  update  the  EA  to  reflect  changes  that 
occurred  during  the  2  years  between  when  the  EA  was  approved  and  when  the 
LCCS  contract  was  awarded.  The  EA  was  prepared  in  September  2001  and 
validated  October  4,  2001.  When  the  PMO  awarded  the  LCCS  contract  on 
January  25,  2004,  the  EA  had  not  been  updated.  The  EA  Manual  states  that 
ongoing  programs  must  be  assessed  periodically  for  their  cost-effectiveness  and 
that  those  assessments  should  include  a  comparison  of  actual  performance  with 
the  approved  project.  To  do  that,  an  update  to  the  program’s  EA  is  often  required. 
Also,  the  EA  Manual  states  that  the  EA  should  include  all  anticipated  costs 
associated  with  each  alternative  over  the  life  of  the  project,  to  include  an  estimate 
of  all  future  costs  through  implementation,  operation,  and  disposal  of  a  project. 
According  to  PMO  officials,  the  decision  to  proceed  with  LCCS  was  based  on  the 
EA.  However,  the  LCCS  contract  that  was  awarded  in  January  2004  included 
costs  that  were  not  reflected  in  the  EA.  Costs  not  reflected  in  the  EA  included 
JV  Contractor  depot  relocation,  technical  data  packages,  and  contractor’s  profits. 
For  example,  the  depot  relocation  cost  of  $11.7  million,  for  the  JV  Contractor  to 
relocate  its  maintenance  facilities  from  Fayetteville,  North  Carolina,  to  Los 
Angeles,  California,  was  included  in  the  contract,  but  the  EA  had  not  been 
updated  to  reflect  that  cost.  PMO  officials  acknowledged  that  the  EA  should  be 
updated  now  that  additional  cost  factors  have  been  identified.  The  PMO  omission 
of  key  cost  elements  from  the  EA  distorts  its  outcome  and,  as  a  result,  the  PMO 
might  not  have  chosen  the  most  economical  option  for  life-cycle  support  for  the 
Javelin  weapon  system. 

Validation.  Although  ODASA-CE  officials  prepared  a  memorandum  on 
October  4,  2001,  “Javelin  Life  Cycle  Support  Concept  Economic  Analysis,” 
stating  that  they  concurred  with  the  EA,  they  did  not  fully  validate  the  EA  for  the 
Javelin  weapon  system.  The  EA  Manual  states  that  validators  should  ensure  that 
assumptions,  constraints,  and  methodology  are  logical,  reasonable,  complete,  and 
well  documented  and  that  conclusions  and  recommendations  are  reasonably 
supported  by  the  analysis.  ODASA-CE  did  not  fully  validate  the  EA  because  the 
PMO  did  not  prepare  an  EA  development  plan  and  did  not  provide  ODASA-CE 
with  all  key  cost  elements.  In  addition,  ODASA-CE  guidance  on  assessing  the  EA 
was  based  on  telephone  calls  and  office  visits  by  the  PMO.  Further,  ODASA-CE 
officials  did  not  verify  the  underlying  cost  data  provided  by  the  PMO. 
ODASA-CE  also  did  not  use  the  detailed  checklist  in  the  EA  Manual  that 
provides  validators  a  general  guide  to  assist  in  the  review  and  validation  process. 
In  addition,  no  formal  report  was  prepared  to  assess  ODASA-CE  objectives  and 
to  document  how  ODASA-CE  conclusions  and  recommendations  were  arrived  at. 
According  to  ODASA-CE  personnel,  their  focus  on  assessing  the  EA  was  to 
ensure  that  it  consisted  of  only  the  necessary  cost  elements  and  that  those  cost 
elements  were  accurate.  Without  the  proper  validation  of  the  underlying  data,  the 
conclusion  regarding  the  overall  validity  of  the  EA  cannot  be  supported. 
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Life-Cycle  Contractor  Support  Alternative 


The  PMO  prepared  an  EA  in  September  2001  and  used  it  as  the  basis  for  choosing 
an  LCCS  alternative  for  the  Javelin  weapon  system.  The  PMO  awarded  the 
Javelin  LCCS  contract  to  the  JV  Contractor  on  January  25,  2004.  However,  the 
EA  was  not  supported  by  an  EA  development  plan,  did  not  address  key  cost 
elements,  had  not  been  updated,  and  had  not  been  fully  validated.  As  a  result,  the 
PMO  might  not  have  selected  the  best  alternative  for  the  life-cycle  support  of  the 
Javelin  weapon  system. 


Management  Actions 


On  January  23,  2004,  USD(AT&L)  issued  a  memorandum,  “Performance  Based 
Logistics  Business  Case  Analysis,”  which  provides  guiding  principles  for 
preparing  a  PBL  business  case  analysis  (BCA).5  A  BCA  is  a  document  in  which 
results  from  various  analyses,  such  as  the  EA,  are  used  to  validate  the  product 
support  strategy.  The  memorandum  requires  that  all  BCAs  be  based  on 
warfighter- stated  performance  requirements  that  are  documented  in  a  PBA.  The 
Military  Departments  were  directed  to  revise  their  PBL  BCA  guidance  to 
incorporate  the  guiding  principles  provided  in  the  memorandum.  A  USD(AT&L) 
March  2004  memorandum,  “Performance-Based  Logistics  and  the  Business  Case 
Analysis,”  states  that  strategic  planning  guidance  requires  the  Services  to 
complete  a  BCA  on  all  new  and  fielded  Acquisition  Category  I  and  II  programs 
by  September  30,  2006. 

In  May  2004,  the  Deputy  Assistant  Secretary  of  the  Army  (Integrated  Logistics 
Support)  issued  the  “U.S.  Army  Implementation  Guide:  Performance-Based 
Logistics,”  which  requires  that  a  BCA  be  prepared  in  implementing  the  PBL 
process.  The  guidance  also  states  that  BCAs  requiring  Army  headquarters 
approval  are  to  be  validated  by  ODASA-CE.  Although  the  Army’s  guidance 
requiring  that  a  BCA  be  prepared  was  issued  in  May  2004,  as  of  August  20,  2004, 
the  Army  was  still  in  the  process  of  developing  guidance  on  how  to  prepare  a 
BCA.  In  addition,  as  of  August  23,  2004,  the  Assistant  Deputy  Under  Secretary 
of  Defense  (Logistics  Plans  and  Programs)  was  in  the  process  of  drafting  a 
revised  comprehensive  handbook  for  PBL  BCAs. 

Because  our  review  showed  that  the  decision  to  award  the  LCCS  contract  to  the 
JV  Contractor  was  not  fully  supported  by  the  EA  or  properly  validated  by 
ODASA-CE,  the  PMO  should  update  the  EA.  Once  updated,  the  EA  should  be 
included  in  the  BCA  that  is  required  by  September  30,  2006,  and  the  BCA  should 
be  validated  by  ODASA-CE.  The  BCA  can  then  be  used  as  the  basis  to  reassess 
decisions  on  continuing  the  PBL  support  strategy  for  the  Javelin  weapon  system. 


5  A  BCA  is  defined  as  a  tool  used  to  manage  business  process  improvement  activities  from  inception 
through  implementation  that  identifies  functional  alternatives  and  presents  economical  and  technical 
arguments  for  carrying  out  alternatives  over  the  life  cycle  of  a  program  to  achieve  stated  business 
objectives  or  imperatives. 
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Management  Comments  on  the  Finding  and  Audit  Response 


Management  Comments.  The  Acting  DUSD(L&MR)  disagreed  with  the  report 
finding  and  stated  that  the  EA  that  was  conducted  by  the  program  manager  and 
approved  by  the  Army  Cost  and  Economic  Analysis  Center,  now  the 
ODASA-CE,  in  October  2001  was  in  accordance  with  existing  DoD  guidance  and 
procedures  for  PBL  that  were  in  place  at  that  tune. 

The  Deputy  Assistant  Secretary  of  the  Army  (Cost  and  Economics)  also  disagreed 
with  the  report  finding  that  the  EA  was  unsupported  and  incomplete  and  stated 
that  his  office  performed  a  comprehensive  review  to  fully  validate  the  Javelin  EA, 
to  include  conducting  interviews  with  PMO  officials  and  analyzing  PMO 
documentation,  hi  addition,  he  stated  that  the  completion  of  an  EA  checklist  and 
an  EA  development  plan  does  not  ensure  an  EA  is  fully  validated. 

Audit  Response.  As  stated  in  this  report,  we  attempted  to  obtain  source 
documentation  from  the  Javelin  PMO  and  ODASA-CE  to  support  the  costs  used 
in  the  Automated  Cost  Estimating  Integrated  Tools  model.  Neither  office  could 
provide  source  documentation  to  support  all  costs  used  in  the  model,  such  as 
spare  part  unit  costs,  mean  time  between  failure,  and  labor  rates.  Further,  the  EA 
Manual  states  that  “it  is  essential  to  adequately  document  the  EA  [and]  that  there 
must  be  sufficient  documentation  of  all  assumptions,  costs,  methodology,  results 
and  data  to  enable  a  person  unfamiliar  with  the  project  to  arrive  at  the  same 
conclusion  as  the  person  who  prepares  it.”  The  EA  Manual  further  states  that  the 
EAs  are  subject  to  many  levels  of  review,  including  the  DoD  Inspector  General, 
adding:  “These  reviewers  may  not  be  as  familial-  with  the  EA  as  the  analyst  that 
prepared  it  and  each  will  critically  analyze  and  pass  judgment  on  the  EAs  validity 
and  adequacy.  For  this  reason  it  is  of  paramount  importance  to  maintain  an 
adequate  audit  trail  to  support  your  conclusions.  The  documentation  must 
provide  an  audit  trail  that  permits  validation  of  all  costs  and  benefits.” 

Without  an  adequate  audit  trail,  we  were  not  able  to  arrive  at  the  same  conclusion 
about  the  EA  as  the  PMO  or  ODASA-CE.  Therefore,  we  could  not  validate  the 
costs  and  benefits  included  in  the  EA.  We  agree  that  an  EA  checklist  and  an 
EA  development  plan  do  not  ensure  a  fully  validated  EA;  however,  if  properly 
used,  those  tools  would  have  ensured  that  all  Javelin  cost  estimates  were 
supported  by  valid  source  documentation  and  that  the  EA  was  performed  based 
on  a  clearly  defined  objective  that  was  measurable,  realistic,  achievable,  and 
results-oriented.  Further,  a  checklist  would  have  provided  assurances  that  the  EA 
could  “stand  on  its  own,”  allow  an  independent  reviewer  to  reach  the  same 
conclusion,  and  ensure  supporting  documentation  was  adequate  for  reviewers  to 
duplicate  costs  and  estimates  used  in  the  EA. 
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Recommendations,  Management  Comments,  and  Audit 
Response 


B.l.  We  recommend  that  the  Project  Manager,  Close  Combat  Weapon 
Systems  Project  Office: 

a.  Update  the  economic  analysis  and  incorporate  it  into  the  business 
case  analysis  for  the  Javelin  weapon  system. 

b.  Prepare  a  business  case  analysis  for  the  Javelin  weapon  system 
and,  based  on  the  results  of  that  analysis,  reassess  the  Javelin  weapon  system 
logistics  support  strategy. 

Management  Comments.  The  DASA(ILS)  partially  concurred,  stating  that  the 
Javelin  EA  was  developed  and  approved  before  the  establishment  of  Defense  and 
Army  guidance  on  BCAs.  He  also  stated  that  the  EA  would  be  updated  to  a  BCA 
when  contract  situations  require  it,  but  that  until  Army  BCA  guidance  is 
developed,  the  Army  would  continue  to  use  the  interim  guide  that  was  prepared 
by  CEAC.  The  Acting  DUSD(L&MR)  stated  that  the  Army  will  be  expected  to 
reassess  the  BCA  and  update  the  EA  to  a  BCA  when  situations  require  it. 

Audit  Response.  Management  comments  are  responsive. 

B.2.  We  recommend  that  the  Assistant  Secretary  of  the  Army  (Financial 
Management  and  Comptroller)  direct  the  Deputy  Assistant  Secretary  of  the 
Army  for  Cost  and  Economics  to  fully  validate  the  business  case  analysis  for 
the  Javelin  weapon  system  and  formally  document  the  results  and 
conclusions  of  that  validation. 

Management  Comments.  The  Deputy  Assistant  Secretary  of  the  Army  (Cost 
and  Economics)  concurred  and  stated  that  upon  receipt  of  the  BCA  for  the  Javelin 
weapon  system,  his  office  will  fully  validate  the  BCA  and  document  the  results  in 
accordance  with  established  procedures. 
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Appendix  A.  Scope  and  Methodology 


We  performed  the  audit  at  the  Close  Combat  Weapon  Systems  Project  Office, 
Aviation  and  Missile  Command,  Huntsville,  Alabama.  We  contacted  personnel  at 
the  Office  of  the  USD(AT&L);  the  Office  of  the  Assistant  Secretary  of  the  Army 
(Acquisition,  Logistics,  and  Technology);  the  Army  Materiel  Command;  the 
Program  Executive  Office,  Tactical  Missiles;  and  ODASA-CE. 

We  assessed  the  adequacy  of  the  Close  Combat  Weapon  Systems  Project  Office 
implementation  of  PBL  by  reviewing  DoD  and  Army  policies  and  regulations 
regarding  responsibilities  and  procedures  for  implementing  PBL  and  reviewed 
FY  2002  quarterly  status  reports.  We  also  reviewed  the  Quadrennial  Defense 
Review  and  DoD  directives  pertaining  to  PBL.  We  reviewed  the  adequacy  of  the 
EA  by  reviewing  Army  guidance  and  cost  documents.  We  assessed  the  LCCS 
contract  for  PBL  provisions. 

We  also  reviewed  the  Army  implementation  schedule  for  PBL  and  interviewed 
PMO  officials  on  the  status  of  implementing  PBL.  Additionally,  we  interviewed 
ODASA-CE  officials  to  discuss  the  validation  process  for  the  EA  and  reviewed 
their  October  4,  2001,  memorandum  that  concurred  with  the  PMO  conclusion  in 
the  EA.  Documents  we  reviewed  were  dated  from  April  1996  through  August 
2004. 

We  performed  this  audit  from  December  2003  through  September  2004  in 
accordance  with  generally  accepted  government  auditing  standards. 

Use  of  Computer-Processed  Data.  We  did  not  rely  on  the  use  of  computer- 
processed  data  to  perform  this  audit. 

Government  Accountability  Office  High-Risk  Area.  The  Government 
Accountability  Office  has  identified  several  high-risk  areas  in  DoD.  This  report 
provides  coverage  of  the  Defense  Weapon  Systems  Acquisition  high-risk  area. 


Management  Control  Program  Review 


DoD  Directive  5010.38,  “Management  Control  Program,”  August  26,  1996,  and 
DoD  Instruction  5010.40,  “Management  Control  Program  Procedures,” 

August  28,  1996,  require  DoD  organizations  to  implement  a  comprehensive 
system  of  management  controls  that  provides  reasonable  assurance  that  programs 
are  operating  as  intended  and  to  evaluate  the  adequacy  of  the  controls. 

Scope  of  the  Review  of  the  Management  Control  Program.  We  reviewed  the 
Close  Combat  Weapon  Systems  Project  Office’s  process  for  reporting  PBL  for 
the  Javelin  weapon  system  and  the  office’s  procedures  for  supporting  and 
completing  the  EA.  We  also  reviewed  the  adequacy  of  management’s  self- 
evaluation  of  those  processes  and  procedures. 
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Adequacy  of  Management  Controls.  We  identified  material  management 
control  weaknesses  within  the  Aimy,  as  defined  by  DoD  Instruction  5010.40. 

The  processes  and  procedures  used  by  the  Army  were  insufficient  to  ensure  that 
PBL  for  the  Javelin  weapon  system  had  been  fully  implemented  and  that  the 
Javelin  EA  was  supported,  complete,  and  fully  validated.  The  recommendations, 
if  implemented,  will  correct  the  conditions  cited.  A  copy  of  the  report  will  be 
provided  to  the  senior  official  responsible  for  management  controls  in  the  Army. 

Adequacy  of  Management’s  Self-Evaluation.  The  Army  did  not  identify  the 
reporting  of  PBL  implementation  and  preparing  and  validating  the  EA  as 
assessable  units  and,  therefore,  did  not  identify  or  report  the  material  management 
control  weaknesses  identified  by  the  audit. 
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Appendix  B.  Prior  Coverage 


During  the  last  5  years,  the  Government  Accountability  Office  (GAO),  the 
Department  of  Defense  Inspector  General  (DoD  IG),  and  the  Navy  have  issued 
12  reports  related  to  implementing  PBL.  Unrestricted  GAO  reports  can  be 
accessed  over  the  Internet  at  http://www.gao.gov.  Unrestricted  DoD  IG  reports 
can  be  accessed  at  http://www.dodig.mil/audit/reports. 


GAO 


GAO  Report  No.  GAO-04-715,  “Defense  Management:  Opportunities  to 
Enhance  the  Implementation  of  Performance-Based  Logistics,”  August  16,  2004 

GAO  Report  No.  GAO-02-1049,  “Contract  Management:  Guidance  Needed  for 
Using  Performance- Based  Service  Contracting,”  September  23,  2002 

GAO  Report  No.  GAO-02-306,  “Defense  Logistics:  Opportunities  to  Improve 
the  Army’s  and  the  Navy’s  Decision-making  Process  for  Weapons  Systems 
Support,” 

February  28,  2002 

GAO  Report  No.  GAO-Ol-618,  “Defense  Logistics:  Air  Lorce  Lacks  Data  to 
Assess  Contractor  Logistics  Support  Approaches,”  September  7,  2001 


DoD  IG 


DoD  IG  Report  No.  D-2004-110,  “The  Military  Departments’  Implementation  of 
Performance-Based  Logistics  in  Support  of  Weapon  Systems,”  August  23,  2004 

DoD  IG  Report  No.  D-2004-021,  “Effectiveness  of  Maintenance  Work  Performed 
Under  Contract  LA4452-01-C-0001  at  Andrews  Air  Lorce  Base,” 

November  19,  2003 

DoD  IG  Report  No.  D-2003-120,  “L/A-18E/F  Integrated  Readiness  Support 
Teaming  Program,”  August  8,  2003 

DoD  IG  Report  No.  D-2002-112,  “Industrial  Prime  Vendor  Program  at  the  Air 
Lorce  Air  Logistics  Centers,”  June  20,  2002 

DoD  IG  Report  No.  D-2000-180,  “Commercial  Contract  for  Total  Logistics 
Support  of  Aircraft  Auxiliary  Power  Units,”  August  31,  2000 
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Navy 


Naval  Audit  Service  Report  No.  N2003-0050,  “Contractor  Logistics  Support 
Oversight,”  May  15,  2003 

Naval  Audit  Service  Report  No.  N2003-0024,  “Contractor  Logistics  Support  at 
the  Naval  Air  Systems  Command,”  January  29,  2003 

Naval  Audit  Service  Report  No.  N2002-0069,  “Contractor  Logistics  Support  at 
the  Space  and  Naval  Warfare  Systems  Command,”  August  8,  2002 
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Appendix  C.  Report  Distribution 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  (Comptroller )/Chief  Financial  Officer 
Deputy  Chief  Financial  Officer 
Deputy  Comptroller  (Program/Budget) 

Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 

Acting  Deputy  Under  Secretary  of  Defense  (Logistics  and  Materiel  Readiness) 
Assistant  Deputy  Under  Secretary  of  Defense  (Logistics  Plans  and  Programs) 
Director,  Program  Analysis  and  Evaluation 
Director,  Defense  Procurement  and  Acquisition  Policy 


J  oint  Staff 

Director,  Joint  Staff 


Department  of  the  Army 

Assistant  Secretary  of  the  Army  (Acquisition,  Logistics,  and  Technology) 
Deputy  Assistant  Secretary  of  the  Army  (Integrated  Logistics  Support) 
Assistant  Secretary  of  the  Army  (Financial  Management  and  Comptroller) 
Deputy  Assistant  Secretary  of  the  Army  (Cost  and  Economics) 

Auditor  General,  Department  of  the  Army 
Commander,  Aviation  and  Missile  Command 

Commander,  Program  Executive  Office,  Tactical  Missiles 

Project  Manager,  Close  Combat  Weapon  Systems  Project  Office 


Department  of  the  Navy 

Naval  Inspector  General 

Auditor  General,  Department  of  the  Navy 


Department  of  the  Air  Force 

Auditor  General,  Department  of  the  Air  Force 


Combatant  Command 

Inspector  General,  U.S.  Joint  Forces  Command 
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Other  Defense  Organizations 

Director,  Defense  Logistics  Agency 


Non-Defense  Federal  Organization 

Office  of  Management  and  Budget 


Congressional  Committees  and  Subcommittees,  Chairman  and 
Ranking  Minority  Member 

Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Aimed  Services 
Senate  Committee  on  Governmental  Affairs 
House  Committee  on  Appropriations 

House  Subcommittee  on  Defense,  Committee  on  Appropriations 
House  Committee  on  Aimed  Services 
House  Committee  on  Government  Reform 

House  Subcommittee  on  Government  Efficiency  and  Financial  Management,  Committee 
on  Government  Reform 

House  Subcommittee  on  National  Security,  Emerging  Threats,  and  International 
Relations,  Committee  on  Government  Refoim 
House  Subcommittee  on  Technology,  Information  Policy,  Intergovernmental  Relations, 
and  the  Census,  Committee  on  Government  Refoim 
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Acting  Deputy  Under  Secretary  of  Defense 
(Logistics  and  Materiel  Readiness)  Comments 


Final  Report 
Reference 


Deputy  under  Secretary  of  Defense  for 

LOGFSTfCS  AND  MATERIEL  READINESS 
3500  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301  3500 


DEC  3  2004 


MEMORANDUM  FOR  PROGRAM  DIRECTOR,  READINESS  AND  LOGISTICS  SUPPORT 
OFFICE  OF  THE  INSPECTOR  GENERAL 


THROUGH:  DIRECTOR,  ACQUISITION  RESOURCES  AND  ANALYSIS 


SUBJECT:  Response  to  DODIG  Draft  Report  D2004LH-0047,  "Report  on  Implementation  of 
Performance-Based  Logistics  for  the  Javelin  Weapon  System,"  October  4,  2004 


I  appreciate  the  opportunity  to  comment  on  the  subject  report.  Although  the  specific 
recommendations  on  the  Javelin  Performance-Based  Logistics  (PBL)  contract  and  performance 
agreements  are  addressed  to  the  Army,  I  would  like  to  provide  comment  since  we  believe  the 
Army  acted  responsibly  in  following  OSD  guidance  that  was  in  place  at  tlie  time  in  conducting 
the  Economic  Analysis  and  awarding  the  PBL  contract 

The  Javelin  weapon  system  represents  one  of  the  earliest  examples  of  the  Army’s  pursuit 
of  PBL  strategies.  The  strategy  pursued  and  employed  as  represented  in  the  contract  awarded  in 
January  2004  was  in  accordance  with  existing  PBL  guidance  and  established  procedures  that 
were  in  place  at  that  time.  The  Army  based  its  decision  to  pursue  PBL  as  a  follow-on  to  the 
existing  support  arrangement  which  was  in  place  since  1996.  As  a  result,  the  strategy  changed 
from  a  typical  Contractor  Logistics  Support  arrangement  based  on  transactions  to  a  Logistics 
Support  arrangement  based  on  performance  outcomes  -  in  this  case,  system  availability.  We, 
therefore,  believe  that  the  Army  should  continue  PBL  reporting  on  the  Javelin.  Wc  agree  that  the 
Army  should  include  incorporation  of  incentives  and  penalties  to  the  PBL  arrangement,  but  only 
when  it  is  prudent  to  do  so,  such  as  when  there  is  cause  for  re-competing  the  support 
arrangement. 

Likewise,  the  economic  analysis  that  was  conducted  by  the  Program  Manager  (PM)  and 
approved  by  the  Army  Cost  and  Economic  Analysis  Center  (CEAC)  in  October  2001  was  in 
accordance  with  existing  OSD  guidance  and  procedures  for  PBL  that  were  in  place  at  that  time. 
As  staLcd  in  the  draft  report,  OSJ>-AT&L  published  guiding  principles  for  preparing  a  Business 
Case  Analysis  (BCA)  in  January  2004.  We  would  expect  that  the  Army  will  reassess  the  need 
for  conducting  a  BCA  for  the  Javelin  and  update  the  economic  analysis  to  a  BCA  when  program 
circumstances/c  hanges  require  an  update. 


Questions  regarding  this  memorandum  may  be  directed  to  Mr.  Anthony  Stampone  at 
703-614-3838  or  at  Ajnthom.5tampoiic@osd.mil. 


cc:  SA(ALT) 


Bradley  Bcrkson 
Acting 


Revised 


21 


Department  of  the  Army  Comments 


DEPARTMENT  OF  THE  ARMY 

OFFICE  OF  THE  ASSISTANT  SECRETARY  OF  THE  ARMY 
ACQUISITION  LOGISTICS  AND  TECHNOLOGY 
DEPUTY  ASSISTANT  SECRETARY  OF  THE  AHMV 
CINYFC RATED  LOGISTICS  SUPPORT! 

103  ARMY  PENTAGON 
WASHINGTON  BG  2IH11HI1H 

November  23.  2004 


SAAL-ZL 


MEMORANDUM  FOR  INSPECTOR  GENERAL,  DEPARTMENT  OF  DEFENSE. 

400  ARMY  NAVY  DRIVE,  ARLINGTON,  VIRGINIA  22202 

SUBJECT:  Army  Comments  to  Draft  DoDIG  Report  on  Im pigmentation  of 
Performance-Based  Logistics  for  the  Javelin  Weapon  System 
(Project  No.  D2004LH-CG47) 


We  have  reviewed  the  subject  draft  report  and  have  provided  our  comments  at  the 
enclosure. 

The  points  d  contact  for  this  action  are  Mr.  Larry  W.  Hill,  DSN  664-7450,  commercial 
(703)  604-7450  ore-mail:  larry.w.tiNt  @ us.army.mil. 


•el* 

integrated  Lopist>cs  Support 


Enclosure 

CF: 

DIRECTOR.  ARMY  AUDIT  AGENCY 
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Final  Report 
Reference 


Army  Comments  to  Draft  DoDIG  Report  on  Implementation  of  Performance- Based 
Logistics  for  the  Javelin  Weapon  System  (Project  No.  D20Q4LH-0G47) 


l .  Recommendation  Al .  Nonconcur.  The  Army  and  OSD  consider  the  Javelin  LCCS 
program  to  be  a  PBL  since  the  performance  outcome  for  the  Javelin  PBL  is  a  90% 
operational  readiness  rate.  This  performance  outcome  fits  the  OSD  definition  of  a  PBL. 
The  Javelin  PBL  strategy  was  approved  by  the  Army  Acquisition  Executive  (AAE)  and 
the  Army  G4  (then  the  DA  DCSLOG)  in  a  13  Mar  02  memorandum.  PBL  has  evolved 
from  the  Section  912  studies  of  Product  Support  Reengineering  and  the  Reduction  of 
Total  Ownership  Costs  (RTOC)  program  which  fcegan  in  1998.  The  Javelin  PBL 
strategy  was  begun  under  the  auspices  of  RTOC,  but  has  migrated  (along  with  the 
migration  of  product  support/RTQC  to  PBL)  to  fit  the  goafs  and  definition  of  PBL. 

2  Recommendation  A2a.  Partially  concur.  The  Javelin  support  strategy  was  approved 
by  the  Army  prior  to  the  majority  of  the  OSD  PBL  policy.  The  Army  considers  Materiel 
Fielding  Agreements  (MFA)  to  be  an  acceptable  form  foe  a  Performance  Based 
Agreement  (PBA).  The  Army  PBL  IPT  currently  has  a  sub-IPT  developing  Army  policy 
on  PBA&.  This  policy  will  state  that  MFAs  can  serve  as  a  PBA  wilh  some  modification  to 
include  additteuai  requirements  such  as  performance  outcomes  and  other  information 
that  would  be  contained  in  a  PBA.  The  intent  of  Army  policy  is  to  allow  flexibility  in 
using  several  existing  documents  to  satisfy  the  PBA  requirement  After  ihe  PBA  policy 
is  approved  and  disseminated,  ms  Javelin  PM  will  be  required  to  update  their  PBAs,  as 
necessary,  Also,  as  the  Javelin  PBL  contract  requirements  change,  the  PBAs  will  be 
updated  as  required. 

3.  Recommendation  A2b.  Partially  concur.  While  the  Army  agreos  that  contract 
incentives/disincentives  are  the  correct  mechanism  to  use  in  PBL  contracts, 
incentives/dislncentrvss  are  the  preferred  approach,  not  the  mandatory  approach.  The 
contractor  is  incenllvized  in  that  the  contact  could  be  terminated  if  the  contractor  tails  to 
meet  the  performance  requirements  in  the  contracl.  When  Javelin  program 
requirements  change  to  require  a  change  in  the  PBL  contract,  Javelin  contract 
incentives/disincenfives  will  be  relooked. 

4,  Recommendation  B1  a  and  b.  Partially  concur.  The  Javelin  PBL  was  approved 
based  on  an  Economic  Analysis  (EA)  whicn  predated  OSD  and  Army  guidance  on 
Business  Case  Analyses  (BCAs).  The  EA  was  validated  by  CEAC.  Therefore,  the 
Army  will  require  the  PM  to  update  the  EA  to  a  BCA  when  program  and  contract 
situations  require  it  Additionally,  an  Army  PBL  IPT  sub-lPT  is  developing  BCA  policy 
and  guidance  for  the  Army  Until  the  Army  BCA  guidance  is  published,  an  interim  Army 
BCA  guide  is  in  effect.  This  interim  guide,  prepared  by  CEAC,  is  an  EA  guide. 
Therefore,  Ihe  Javelin  did  comply  with  the  OSD  and  Army  guidance  in  effect  at  the  lime 
the  Javelin  PBL  strategy  was  being  developed, 

5  Recommendation  B2.  Concur,  When  program  situations  require  the  current  EA  to 
be  updated  to  a  BCA,  CEAC  will  be  required  to  independently  validate  the  BCA. 
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DEPARTMENT  OF  THE  ARMY 

OFFICE  OF  THE  ASSISTANT  SECRETARY  OF  ThE  ARMY 
FINANCIAL  MANAGEMENT  AND  COMPTROLLER 
1M  ARMY  PENTAGON 
WASHINGTON,  DC  2031D-O109 


December  3,  2004 


SAFM-CEA-WS 


MEMORANDUM  FOR  DOD  INSPECTOR  GENERAL,  ATTN:  PROGRAM 

DIRECTOR,  READINESS  AND  LOGISTICS  SUPPORT, 
400  ARMY  NAVY  DRIVE,  ARLINGTON,  VIRGINIA 
22202-4704 

SUBJECT:  Office  of  the  Deputy  Assistant  Secretary  of  the  Army  for  Cost  and 
Economics  Comments  to  IG  Report  on  Implementation  of  Performance-Based  Logistics 
for  the  Javelin  Weapon  System  (Project  No.  D2004LH-0047) 


1 .  This  is  in  response  to  the  DODIG  4  Oct  04  memorandum,  “Implementation  of 
Perform  ance-Based  Logistics  for  the  Javelin  Weapon  System,”  Wc  are  providing 
comments  on  the  economic  analysis  portion  of  the  report. 

2.  Overview.  Die  DODIG  reviewed  the  Javelin  program^  implementation  of 
performance-based  logistics  to  include  the  performance  agreements,  contract,  schedule, 
and  economic  analysis.  Wc  strongly  disagree  with  the  report  conclusion  that  the 
economic  analysis  was  unsupported  and  incomplete, 

3.  Report  Summary  and  Findings.  Die  Executive  Summary  of  the  report  states  that,  “the 
Program  Management  Office  (PMO)  decision  to  award  die  life  cycle  contractor  support 
(LCCS)  contract  to  the  Joint  Venture  (JV)  was  based  on  an  economic  analysis  (EA)  that 
was  unsupported  and  incomplete.”  Specific  Findings  in  Section  R  titled,  “Economic 
Analysis”  state,  "'Dial  the  EA  was  unsupported  and  incomplete  because  the  PMO  and  the 
Army  Cost  and  Economic  Analysis  Center  (CEAC)  did  not  adhere  to  requirements  for 
preparing  and  validating  it.  Specifically,  the  PMO  did  not  ensure  that  the  EA  had  an  EA 
development  plan,  had  a  clear  audit  trail  and  had  been  updated  with  key  cost  elements. 
Further,  CEAC  had  not  fully  validated  the  EA."” 

4.  Comments.  The  Office  of  the  Deputy  Assistant  Secretary  of  the  Army  for  Cost  and 
Economics  (ODASA-CE),  formerly  known  as  CEAC,  performed  a  comprehensive 
review  to  fully  validate  the  Javelin  EA.  We  will  address  each  of  the  highlighted  findings 
below. 

a  The  EA  was  unsupported  and  incomplete.  The  BA  review  of  the  methodology 
and  factors  included  interviews  and  documentation  reviews  with  PMO  cost  analysts  who 
performed  the  EA.  Further,  the  context  of  each  line- within  the  Automated  Cost 
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Estimating-  Integrated  I  ools  (ACEI 1  j  mode],  the  Army's  standard  cost  model*  containing 
the  EA  was  reviewed  and  validated  to  include  the  assump lions,  constraints,  equations, 
background  of  each  variable  and  factor,  and  documentation  for  applicability  and  balance 
throughout  the  validation  process. 

b.  The  PM O  did  not  ensure  that  the  EA  had  an  EA  development  plan  [EADPJ, 
had  a  clear  audit  trail  and  had  been  updated  with  key  cost  elements.  The  preparation 
of  an  EADP  docs  not  ensure  a  fully  validated  BA,  ODASA-CE  was  a  member  of  the 
Javelin  Integrated  Product  Team  (IPT)  from  the  beginning.  The  IPT  comprised  of 
functional  personnel  from  logistics  and  cost  areas  as  well  as  legal  and  contracting, 
developed  die  overall  EA  review  process.  The  EADP  is  not  a  requirement  and  the  overall 
analysis  plan  and  options  were  developed  and  approved  by  the  IPT.  Since  a  list  of 
missing  “key  cost  elements11  was  not  provided  in  the  DODIG  report,  it  is  impossible  to 
address  concerns  in  this  area. 

c*  CEAC  had  not  fully  validated  the  EA.  The  Javelin  EA  was  validated  through  an 
extensive  review  of  the  EA  cost  documentation  in  the  ACEIT  file  in  coordination  with 
PMO  cost  analysis.  This  included  a  review  of  the  source  and  derivation  of  all  costs 
contained  in  the  EA.  The  report  also  noted  that  analysts  did  not  use  the  checklist 
provided  in  the  EA  Manual  and  there  was  no  formal  report.  These  two  issues  are  covered 
below. 

Checklist.  The  checklist  referenced  in  the  DA  EA  Manual  at  Appendix  M  is  a  set  of 
questions  developed  as  a  tool  to  guide  the  preparation,  review,  and  validation  of  EAs. 

This  list  is  not  intended  nor  required  to  be  a  piece  of  documentation  for  the  validation  of 
an  EA,  The  ODASA-CE  validation  of  the  BA  did  cover  the  necessary  questions  inherent 
in  ensuring  (hat  the  EA  was  complete,  comprehensive,  and  properly  conducted  according 
to  the  guidelines  in  the  DA  EA  Manual.  Also,  at  that  time,  there  was  no  unique  guidance 
for  PBL  validations. 

Formal  Report.  DA  approved  cost  estimates  arc  documented  in  the  Cost  Analysis  Brief 
(CAB)  and  the  ACEIT  file  of  the  system  cost  estimate.  This  documentation  was 
considered  to  be  sufficient  by  the  CEAC  Director  once  the  EA  was  determined  to  he 
valid,  ft  is  standard  practice  to  include  all  documentation  as  an  automated  file  within  the 
ACEIT  cost  model.  Therefore,  a  paper  report  was  not  required  which  was  the  Director's 
prerogative  as  the  proponent  of  the  DA  EA  Manual. 

5.  Report  Recommendation.  The  Army's  draft  PFtl\  Guidance  will  require  periodic 
updates  of  analyses  used  to  determine  the  supportability  approach,  Wc  concur  with  the 
draft  report  Recommendation  B.2.  Upon  receipt  of  the  business  case  analysts  for  the 
Javelin  weapon  system,  ODASA-CE  will  fully  validate  the  business  case  analysis  for  the 
Javelin  weapon  system  and  document  the  results  in  accordance  with  established 
procedures. 


2 


25 


6,  The  process  used  by  OD  ASA-CE  to  validate  the  EA  based  on  the  program  definition 
at  the  time  of  review  was  a  hands-on  and  in-depth  review  that  was  conducted  with  the 
PMO  analyses.  The  validated  ACEIT  file  serves  as  formal  documentation  of  the  effort, 
and  the  ODASA-CE  memorandum  confirms  validation  and  notes  discrepancies.  We 
hope  that  this  clarifies  the  process  and  procedures  used  by  ODASA-CE.  We  appreciate 
the  opportunity  to  comment  on  the  draft  report.  Questions  regarding  our  comment  may 
he  addressed  to  Mr.  Sean  Vessey,  Chief,  Weapon  Systems  Division,  at  (703  )  601-4138. 


Stephen  T.  Bagby 

Deputy  Assistant  Secretaiy  of  the  Army 
(Cost  and  Economics) 

CF: 

USD(AL&T) 

DIRECTOR,  ARMY  AUDIT  AGENCY 
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